AI Weekly: Novel architectures could make large language models more scalable

Large language models are currently difficult to scale. But this could change with an architecture called mixture of experts.